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Abstract 

Background: The Health Information Technology for Economic and Clinical Health (HITECH) Act subsidizes 
implementation by hospitals of electronic health records with computerized provider order entry (CPOE), which 
may reduce patient injuries caused by medication errors (preventable adverse drug events, pADEs). Effects on 
pADEs have not been rigorously quantified, and effects on medication errors have been variable. The objectives of 
this analysis were to assess the effectiveness of CPOE at reducing pADEs in hospital-related settings, and examine 
reasons for heterogeneous effects on medication errors. 

Methods: Articles were identified using MEDLINE, Cochrane Library, Econlit, web-based databases, and bibliographies 
of previous systematic reviews (September 2013). Eligible studies compared CPOE with paper-order entry in acute care 
hospitals, and examined diverse pADEs or medication errors. Studies on children or with limited event-detection 
methods were excluded. Two investigators extracted data on events and factors potentially associated with 
effectiveness. We used random effects models to pool data. 

Results: Sixteen studies addressing medication errors met pooling criteria; six also addressed pADEs. Thirteen studies 
used pre-post designs. Compared with paper-order entry, CPOE was associated with half as many pADEs (pooled risk 
ratio (RR) = 0.47, 95% CI 0.31 to 0.71) and medication errors (RR = 0.46, 95% CI 0.35 to 0.60). Regarding reasons for 
heterogeneous effects on medication errors, five intervention factors and two contextual factors were sufficiently 
reported to support subgroup analyses or meta-regression. Differences between commercial versus homegrown 
systems, presence and sophistication of clinical decision support, hospital-wide versus limited implementation, and US 
versus non-US studies were not significant, nor was timing of publication. Higher baseline rates of medication errors 
predicted greater reductions (P< 0.001). Other context and implementation variables were seldom reported. 

Conclusions: In hospital-related settings, implementing CPOE is associated with a greater than 50% decline in pADEs, 
although the studies used weak designs. Decreases in medication errors are similar and robust to variations in important 
aspects of intervention design and context. This suggests that CPOE implementation, as subsidized under the HITECH Act, 
may benefit public health. More detailed reporting of the context and process of implementation could shed light on 
factors associated with greater effectiveness. 

Keywords: Medical order entry systems, Drug toxicity/prevention and control, Hospitals, Adverse drug event, 
Medication error 
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Background 

The Health Information Technology for Economic and 
Clinical Health (HITECH) Act of 2009 incentives the 
adoption of health information technology by US hospitals. 
This Act, part of the American Reinvestment and Recovery 
Act, allocates up to $29 billion over 10 years for the imple- 
mentation and 'meaningful use' of electronic health records 
by hospitals and healthcare providers [1]. Hospitals that sat- 
isfy meaningful use criteria can receive millions of dollars. 
Implementing computerized provider order entry (CPOE) 
with clinical decision support systems (CDSS) that check 
for allergies and drug-drug interactions is one of several 
basic (Stage 1) criteria for meaningful use by hospitals [2]. 
As of 2008, approximately 9% of general acute care hospi- 
tals had at least basic electronic health record (EHR) sys- 
tems including CPOE for medications. By 2012, 44% had 
such systems, specifically, 38% of small, 47% of medium, 
and 62% of large hospitals [3]. Thus, despite the financial 
incentives, about half of small and medium hospitals and 
almost 40% of large hospitals had not adopted CPOE with 
CDSS in the most recent survey. 

The primary potential benefit of adopting CPOE is redu- 
cing patient injuries caused by medication errors, called 
preventable adverse drug events (pADEs) [4-6]. Counter- 
balancing this is concern about unintended adverse conse- 
quences [7-9], including increases in medication errors 
and even mortality, which have been detected in some 
hospitals after implementation of CPOE [10,11]. To date, 
no systematic review has examined net effects on pADEs, 
the primary outcome of interest for this intervention. Pre- 
vious reviews have, instead, focused almost exclusively on 
an intermediate outcome, medication errors. However, not 
all medication errors pose an equal risk of causing injury. 
Errors in timing, for example, are generally less risky than 
giving a medication to the wrong patient. Many commonly 
used medications, such as anti-hypertensives and antibiotics, 
have sufficientiy long half-lives that receiving a dose an hour 
or two late has littie clinical effect. By contrast, receiving an 
anti-hypertensive or antibiotic intended for someone else 
poses risks of low blood pressure or an allergic reaction. In 
one study at six hospitals, only about 20% of medication er- 
rors led to pADEs [12]. Thus, the effect of CPOE on the pa- 
tient outcome of pADEs is an important clinical and policy 
question that has remained unanswered, until now. 

In addition to focusing on medication errors rather than 
pADEs, previous systematic reviews have reached conflicting 
conclusions about the effects of CPOE on medication errors 
in acute care settings. Some have concluded that CPOE re- 
duces errors, whereas others argue that net effects remain 
uncertain [4,5,13-42]. This controversy stems, in part, from 
the fact that the association between CPOE implementation 
and medication errors has exhibited substantial heterogen- 
eity across primary studies [37]. Three basic types of factors 
could explain such variability: intervention factors, such as 



differences in how the intervention is designed and imple- 
mented; contextual factors, such as differences in patient 
populations and settings; and methodological factors, such 
as differences in study design and execution [43] . 

Uncertainty about the effects of CPOE on patient out- 
comes and its variable effects on medication errors may 
contribute to the reluctance of some hospitals and physi- 
cians to adopt CPOE, despite the financial incentives avail- 
able via HITECH. Consequently, our primary objective in 
this study was to quantitatively assess the effectiveness of 
CPOE at reducing pADEs in hospital-related acute care set- 
tings. Our secondary objective was to identify factors con- 
tributing to variability in effectiveness at reducing medication 
errors. This analysis is timely as several studies have been 
published recently and, therefore, were not included in previ- 
ous reviews and meta-analyses [4,13,34,37,41], enabling us to 
examine effects on pADEs and reasons for heterogeneity. 

Methods 

We adhered to recommendations in the Preferred Report- 
ing Items for Systematic Reviews and Meta-Analyses 
(PRISMA) Statement [44,45], including developing the 
protocol before undertaking the analysis. 

Data sources and searches 

First, we developed search strategies for eight databases: 
MEDLINE; Cochrane Library; Econlit; Campbell Collab- 
oration; the Agency for Healthcare Research and Quality 
(AHRQ) Health Information Technology Library, Health 
Information Technology Bibliography, Health Informa- 
tion Technology Costs and Benefits Database Project, 
and PSNET; Information Service Center for Reviews and 
Dissemination at the University of York; Evidence for Policy 
and Practice Information and Coordinating Centre (EPPI- 
Centre), University of London; Oregon Health Sciences 
Searchable CPOE Bibliography; and Health Systems 
Evidence, McMaster University. A number of search 
terms, such as 'order entry' and 'electronic prescribing' 
(see Additional file 1), were chosen and strategies de- 
veloped, in part based on a search strategy published 
by Eslami et al. [4]. 

We used this strategy to search the eight databases 
for systematic reviews of CPOE or CDSS that might 
contain potentially relevant primary studies (last updated 
September 23, 2013) (Figure 1). Next, we used the same 
strategy to search the eight databases for potentially rele- 
vant primary studies that were published after two large 
previous systematic reviews on CPOE (January 1, 2007 
to September 23, 2013) [4,13]. In addition, we hand- 
searched nine websites (AHRQ HIT Library, AHRQ 
PSNET, National Patient Safety Foundation, Joint 
Commission, Leapfrog Group, Micromedex, Institute 
for Healthcare Improvement), the Web of Science, and 
bibliographies of other publications known to us. 
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PRISMA 2009 Flow Diagram 



Records identified through 
database searching 
(n = 2,116) 



Additional records identified 
through other sources 
(n = 2,775) 



Records after duplicates removed 
(n = 4,575) 



Records screened 
(n = 4,575) 



Full-text articles assessed 
for eligibility 
(n = 93) 



Studies included in 
qualitative synthesis 
(n = 19) 



Studies included in 
quantitative synthesis 
(meta-analysis) 
(n = 16) 



Figure 1 Summary of evidence search and selection. 



Records excluded 
(n = 4,482) 



Full-text articles excluded, 
with reasons 
(n = 74) 
Did Not Test CPOE: 32 
Setting: 9 
Outcome: 24 

Methods, Event Detection: 6 
Methods, Dissimilar Study 
Populations: 1 
Duplicate Publication: 2 



Study selection 

We included peer-reviewed studies, regardless of lan- 
guage or design, if they compared CPOE with paper- 
order entry and examined either of our two primary 
outcomes, rates of pADEs or medication errors, across 
a variety of clinical conditions. Eligible settings included 
adult medical or surgical wards, adult medical or surgi- 
cal intensive care units (ICUs), emergency depart- 
ments, or the entire hospital. To reduce unwarranted 
variability due to contextual and methodological factors, 
we excluded studies that were from non-hospital settings; 
that addressed events limited to specific conditions (for ex- 
ample, infections) or types of errors (for example, allergy 
alerts); or that compared events in highly dissimilar pa- 
tient care units. As minimum criteria for study quality, 
we excluded studies that did not describe methods for 
detecting medication events, or that used incident 
reporting alone, which detects 0.2 — 6% of events [46]. 
We also excluded pediatric studies because including 



them would increase heterogeneity: children comprise 
only 6% of hospitalized patients whereas ADEs dispro- 
portionately affect older adults [12,47,48]. 

Two investigators independently screened the article 
titles and then abstracts for eligibility. We obtained 
full-text articles when either investigator found the ab- 
stract (or title, if the abstract was unavailable) poten- 
tially eligible. Disagreements about the eligibility of 
full-text articles were resolved by consensus, with a 
third investigator participating for ties. 

Data extraction and quality assessment 

We defined pADEs as injuries to patients due to 
medication errors. Medication errors were defined as 
errors in the process of prescribing, transcribing, dis- 
pensing, or administration of a medication, which had 
the potential to or actually did cause harm. To focus 
on errors involving relatively higher risk, we excluded, 
when reported, 'errors' described as having no or 
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almost no potential for harm as well as incomplete or 
illegible orders, disallowed abbreviations, disallowed 
drug names, and medications given at the wrong time 
(see Additional file 1). 

Two investigators independently extracted data from 
each study using a standardized form (see Additional 
file 1). Disagreements were resolved by consensus, with 
a third investigator adjudicating ties. Extracted elements 
included numbers of pADEs and medication errors meet- 
ing study definitions, units of exposure to risk of pADEs 
or medication errors (for example, number of orders, dis- 
pensed doses, admissions, or patient days). When studies 
reported rates or proportions rather than these elements, 
variance could not be estimated, so the studies could not 
be included in pooled effect calculations and thus we 
qualitatively summarized their results instead. 

From the studies included in the pooled analysis of 
medication errors, we extracted several elements related 
to intervention design and implementation, context, and 
study methods. Elements related to intervention design in- 
cluded: CPOE developer (homegrown versus commercial); 
and presence or absence of CDSS, CDSS sophistication 
(basic, moderate, or advanced; see Table 1 for definitions). 
When information about the system developer and CDSS 
were missing from the published article, we contacted the 
original authors. 

Elements related to implementation were based on 
an AHRQ report addressing context-sensitive patient 
safety practices, including CPOE. These included: fac- 
tors influencing the decision to adopt, factors facilitating 
implementation, and aspects of implementation described 
in the studies, as well as timing, extent of implementation 
(limited number of units versus hospital-wide), and 
whether use was mandatory (see Additional file 1 for de- 
tails) [72]. 

Contextual elements included setting/population (type 
of clinical unit within the hospital, academic status, public 
versus private hospital, hospital size, country, primary lan- 
guage in country, payer mix), and baseline proportion of 
hospitalizations affected by medication errors. 

Methodological elements included type of study design, 
event detection methods, items related to study quality 
(adapted from relevant reporting criteria in the Standards 
for Quality Improvement Reporting Excellence; SQUIRE) 
[73], and funding source. 

Data synthesis and analysis 

Using the DerSimonian-Laird random effects model [74] , 
we conducted meta-analyses for two outcomes (pADEs 
and medication errors) for all eligible studies combined, 
and for different subgroups of studies as described below. 
For each eligible study and outcome measure, we calcu- 
lated a risk ratio (RR) as the number of events per unit of 
exposure in the CPOE group divided by events per unit of 



exposure in the paper-order entry group. Units of expos- 
ure varied across studies. If a study provided more than 
one unit of exposure, we selected the unit most commonly 
used in the included studies. 

Within each meta-analysis, we tested the heterogeneity of 
the log- transformed RRs using Q and I 2 statistics [75]. Het- 
erogeneity was present when the I 2 statistic was 50% or 
more and the P-value for the Q statistic was 0.05 or less. 

We conducted two sensitivity analyses, removing one 
study at a time from each meta-analysis to assess the in- 
fluence of each individual study, and testing whether 
the choice of units of exposure affected results. To assess 
publication bias, we examined funnel plots, Begg and 
Mazumdar's rank correlation test, and Egger's regression 
intercept test [76] . 

Intervention design and implementation, contextual, and 
methodological factors 

A priori, we identified nine factors that might be associ- 
ated with heterogeneity in medication errors across stud- 
ies. Intervention design factors included type of CPOE 
developer (homegrown versus commercial), presence or 
absence of CDSS, and sophistication of CDSS (basic, mod- 
erate, or advanced). Intervention implementation factors 
included scope of implementation (hospital-wide versus 
limited) and timing of CPOE implementation (year CPOE 
was implemented or, if missing, the year the study was 
published). Contextual factors included country (US ver- 
sus non-US) and baseline proportion of hospitalizations 
affected by medication errors. Methodological design fac- 
tors included study design (pre-post versus other designs) 
and event detection methods (pharmacist order review 
versus more comprehensive methods). For each discrete 
factor, we conducted a subgroup analysis when there were 
at least three studies per subgroup, for example, pre/ 
post design versus other design. For each continuous 
factor, we conducted a meta-regression using the factor as 
the sole predictor. In each meta-regression, we pooled 
log-transformed RRs, and presented the pooled results on 
the original RR scale. 

Pooled meta-analyses were conducted using Compre- 
hensive Meta-analysis, V2 (Biostat, Englewood, NJ, USA); 
meta-regression analyses were conducted in STATA (V13) 
(StataCorp LP, College Station, TX, USA). 

Results 

We screened 4,891 potentially eligible records, including 
the bibliographies of 32 systematic reviews on CPOE or 
CDSS [4,5,13-42], and then examined 93 full-text articles 
on CPOE. Of these 93 full-text articles, 74 were excluded: 
32 did not test the effectiveness of CPOE, 3 addressed non- 
hospital settings, 6 addressed pediatric settings, 5 used inci- 
dent reporting alone to detect events, 1 did not describe 
event detection methods, 16 addressed outcomes other 



Table 1 Characteristics of included studies 



Reference Country Number of Financial Type of Setting in Baseline Developer of 

hospitals status hospital hospital error rate, % a CPOE system 

(bed size) 



Use of CDSS b 
CPOE 



Study 
design 



Event detection 
methods c 



Bizovi ef al, 
2002 [49] 

Franklin ef al, 
2009 [50-52] 



Shawahna 
ef al, 201 1 [53] 

Shulman ef al, 
2005 [54] 



Leung ef al, 
2012 [11] 

Wess ef al, 
2007 [55]' [56] 



Taylor ef al, 
2002 [57] e 

Barron ef al, 
2006 [58] 

Bates ef al, 
1 998 [59] 



USA 
UK 

Pakistan 
UK 

USA 
USA 

USA 
USA 
USA 



(560) 



Public 



Academic 



6 (1 00 to 
300 each) 

2 (665 
and 555) 



3 (1000 
in total) 



(525) 



(726) 



Van Doormal The 2(1300 

ef a!., 2009 [60] Nether-lands and 600) 



Westbrook 
ef al, 2012 [61] 



Weant ef al, 
2007 [62] e 



Australia 



USA 



2 (400 
and 326) 



(489) 



(Probably Academic 
public) 



ED 



Genera 
surgery ward 



1 (1280) Public Academic 2 medical wards 
1 (-) Public Academic General ICU 



Community Hospital-wide 

Private Academic, General surgery, 
Community Orthopedic/ 

neurosurgical units 

Private Academic Hospital-wide 



Private Academic Hospital-wide 



Private Academic 2 medical and 2 
surgical wards, 
2 ICUs 

Academic 2 medical wards 
at each hospital 



Academic 4 medical wards 
at one hospital; 
1 cardiology and 1 
psychiatry unit at 
the other hospital 

Public Academic Neurosurgical ICU 



3.6 (visits) 



64.9 



83.8 
41.1 



42.3 



1 0/1 



Commercial 
(EmSTAT; 
CyberPlus) 

Commercial 
(ServeRx V.1:13; 
MDG Medical) 



Homegrown 

Commercial 
(QS 5.6 Clinical 
Information 
System; GE 
Healthcare) 

Commercial 
{not stated) 

Commercial 
(LastWord 8 ; GE, 
formerly IDX) 

Not stated 



Mandatory None d 

Mandatory None 

Mandatory None 

Not stated None 

Not stated Present 



Mandatory Present 
(n = 1) and 
voluntary 
(n = 1) 

Not stated Present 



Homegrown Mandatory Basic 



Homegrown Mandatory Basic 



Mandatory Basic 



Commercial 
(Medicator"; iSoft), 
Partly Homegrown 
(Theriak 9 ), 

Commercial (Millenium Exceptions Basic 
Power Orders; Cerner allowed 
and MedChart; iSoft) 



Pre/post 
Pre/post 

Pre/post 
Pre/post 

Pre/post 
Pre/post 

Pre/post 
Pre/post 
Pre/post 
Pre/post 



Differences 
in differences 



Not stated Not stated Moderate Pre/post 



Routine pharmacist 
review of medication 
orders 

Routine pharmacist 
review of medication 
orders, medical record 
review, and incident 
reporting 

Medical record 
review (R) 

Routine pharmacist 
review of medication 
orders 



Medical record and 
order review (B, R) 

Routine pharmacist 
review of medication 
orders, with changes 
signed by MD 

Quarterly review 

of subset of 
medication orders 

Routine pharmacist 
review of medication 
orders 

Medical record 
review and other 
means (B, R) 

Medical record 
and order review 



Routine pharmacist 
review of medication 
orders (R) 



Routine pharmacist 
review of medication 



3 r? 



o' ? 



-a 

a; 

ID 



Table 1 Characteristics of included studies (Continued) 



Bates ef a/., 
1 999 [63] 



Colpaert ef a/., 
2006 [64] 



Mahoney ef a/., 
2007 [65] 



Oliven ef a/., 
2005 [66] 

Igboechi ef a/., 
2003 [67] e 



Aronsky ef a/., 
2007 [68,69] 



Mendendez 
ef a/., 2012 [70] 



USA 



(700) 



Private Academic 



Academic 



USA 



Israel 

USA 

USA 
Spain 



2 (247 
and 719) 



1 (450) 
1 (350) 

1 (658) 



Private 



(200) 



2 medical 
wards and 
1 ICU 

3 units within 
a surgical ICU 



Private Academic Hospital-wide 



Public Academic Pulmonary service 



Private Community Hospital-wide 



Academic 



ED 



Academic Hospital-wide 



47.3 



98.0 



62. 1 



99.8 (visits) 



5.0 



Homegrown 



Commercial 
(Centricity Critical 
Care Clinisoft; GE 
Healthcare Europe) 

Commercial 
(Siemens Medical 
Solutions CPOE; 
Siemens Medical 
Solutions Health 
Services Corp) 

Homegrown 



Mandatory Moderate Pre/post 



Mandatory Moderate Comparison of 
similar units 



Exceptions Moderate 
allowed 



Pre/post 



Not stated Moderate 
Mandatory Moderate 



Commercial 
(Ulticare System 
Database; Per Se 
Technologies 

Homegrown (WizOrder, Not stated Advanced d 
later commercial-ized 



Compare 
similar units 

Pre/post 



Pre/post 



as Horizon Expert 
Orders; McKesson) 

Commercial 
(Selene; Siemens) 



Not stated Advanced d Pre/post 



orders, incident 
reporting 

Medical record 
and order review 
plus other means 

Routine pharmacist 
review of medication 
orders (B) 

Routine pharmacist review 
of medication orders, with 
changes accepted by 
MD; incident reporting 



Medical record 
and order review 

Routine pharmacist review 
of medication orders 



Routine pharmacist review 
of medication orders 



Trigger tool medical 
record review, incident 
reporting, and other means 



CDSS, Clinical Decision Support Systems; CPOE, computerized provider order entry; ED, emergency department; ICU, intensive care unit. 
Percentage of hospitalizations (or emergency department visits, where noted). 

b None = no clinical decision support system; basic = checks for drug-allergy and drug-drug interaction; moderate = basic plus at least one additional clinical decision support function; advanced = moderate plus additional 
capabilities [71]. 

C T = paper described training of reviewers; B = paper described blinding of reviewers to baseline versus CPOE conditions; R = Paper described methods for assessing reviewer reliability. If none of symbols appear, these 
were not described. 

information on CDSS obtained by contacting authors. 

e Omitted from pooled effect calculations due to lack of data related to estimating variance. 



3 r? 



o' ? 



-a 

a; 

IO 

n> 
o 
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than medication errors or pADEs (for example, workflow 
or cost), 5 addressed errors limited to specific conditions, 1 
addressed specific types of errors, 2 (1 in French) addressed 
errors that were excluded because they posed a lower risks 
of harm, 1 (in Spanish) compared event rates in dissimilar 
clinical units (obstetric and oncology), and 2 were dupli- 
cate publications of articles meeting the selection criteria 
(Figure 1; see Additional file 1, Additional file 2). 

The remaining 19 original articles met the selection 
criteria and addressed medication errors; 7 of these also 
addressed pADEs (Table 1) [11,49,50,53-55,57-68,70]. Of 
these 19 studies, 3 omitted the data needed to estimate 
variance and, therefore, were excluded from pooled ef- 
fect calculations, resulting in 16 eligible studies, includ- 
ing 6 that addressed pADEs [57,62,67]. 

Of the 16 studies, half were based in the US, including 
two in community hospitals [11,55]. Thirteen studies 
used pre/post designs [11,49,50,53-55,58-60,63,65,68,70], 
two compared similar units within a hospital during the 
same time period [64,66], and one compared changes 
over time between intervention and control units (differ- 
ences in differences design) [61]. Definitions of medica- 
tion errors and the methods used to detect them varied 
across studies (see Additional file 1). Seven studies iden- 
tified events using data from routine pharmacist review 
of medication orders [49,54,55,58,61,64,68]. One study 
provided information on reviewer training [11], three on 
blinding of reviewers [11,59,64], and none on reliability. 
The baseline percentage of hospitalizations affected by 
medication errors ranged from 3.6% [49] to 99.9% [60]. 

Nine studies assessed commercially developed CPOE 
systems [11,49,50,54,55,61,64,65,70], six evaluated home- 
grown systems [53,58,59,63,66,68], and one examined both 
[60]. No two studies assessed the same commercial sys- 
tem. CDSS was present in twelve studies [11,55,58-61, 
63-66,68,70], and absent in four [49,50,53,54]; we con- 
tacted and obtained responses from authors for three of 
the studies (Table 1). 

For all but one study [58], most of the desired infor- 
mation on implementation was missing (see Additional 
file 1). Based on the information that was reported, ten 
studies described the use of CPOE as mandatory at one 
or more sites [49,50,53,55,58-61,63,64]. CPOE was im- 
plemented hospital-wide in four studies [11,58,65,70], 
in the emergency department in two studies [49,68], 
and in a limited number of inpatient units in the rest. 
Four studies were conducted in complex organizations 
with facilities in multiple communities [55,59,63,65], an- 
other study was in a large hospital with affiliated clinics 
[49], and another was in community hospitals [11]. Past 
experience with information technology was reported in 
seven studies [49,50,55,58,59,63,65]. Three studies re- 
ported that organizational leadership influenced the adop- 
tion decision [55,58,65], and four stated that staff training 



and education facilitated implementation [53,54,58,66]. 
One study mentioned the role of staff time to learn CPOE, 
a person to lead implementation, extensive project man- 
agement, an implementation timeline, teamwork, and pa- 
tient safety culture related to CPOE [58]. Another study 
described the effects of having a responsible person, local 
tailoring, and teamwork [65]. 

The three studies omitted from the pooled analysis due 
to lack of variance estimates were similar to the included 
studies. They were conducted in the US in medium to 
large hospitals, including one in a community hospital. 
One study evaluated a commercially developed system 
[67]; the other two did not report the developer. Two 
studies included CDSS [57,67]. All three used pre/post de- 
signs, one detected events using pharmacist review of 
medication orders [67], and none reported reviewer train- 
ing, blinding, or reliability. These studies also did not re- 
port implementation context or processes in detail [62,67], 
except for one, which discussed financial considerations 
and leadership [57]. 

Primary outcome: preventable adverse drug events 

Of the 19 studies, 7 assessed pADEs [11,59,60,62-64,70]. 
For the six studies in the pooled analysis, RRs ranged from 
0.17 to 0.81. Overall, CPOE was associated with about half 
as many pADEs as paper-order entry (pooled RR = 0.47, 
95% CI 0.31 to 0.71). Studies were heterogeneous (I 2 = 
69%) (Figure 2). Serial removal of each study did not sub- 
stantially influence results (pooled RR range 0.40 to 0.58). 
There was no evidence of publication bias using a funnel 
plot, or Begg and Mazumdar's test (see Additional file 1). 
For one study excluded from the pooled analysis due to 
lack of data on variance, we calculated an RR of 0.11 [62]. 

Secondary outcome: medication errors 

All 19 studies meeting selection criteria assessed medica- 
tion errors [11,49,50,53-55,57-68,70]. Across the 16 stud- 
ies eligible for the pooled analysis, RRs ranged from 0.16 
to 2.08. The pooled estimate showed that medication er- 
rors were approximately half as common when providers 
used CPOE than when they used paper-order entry 
(pooled RR = 0.46, 95% CI 0.35 to 0.60). The studies were 
highly heterogeneous (7 2 = 99%) (Figure 3). Results were 
robust to serial removal of each individual study (pooled 
RR range 0.42 to 0.49), and to selection of an alternative 
unit of exposure in the four studies where that was pos- 
sible (pooled RR = 0.45, 95% CI 0.34 to 0.59). There was 
no evidence of publication bias using a funnel plot, or 
Begg and Mazumdar's test (see Additional file 1). 

Two studies included in the pooled analysis reported 
increases in medication errors after the introduction of 
CPOE, however, both also reported statistically signifi- 
cant decreases in preventable adverse drug events 
[11,70]. A third study, excluded due to lack of data on 
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Figure 2 Meta-analysis: relative risk of preventable adverse drug events using computerized provider order entry (CPOE) versus 
paper-order entry in hospital acute care settings. Units of exposure: *1,000 patient days; Admissions. 
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Figure 3 Meta-analysis: relative risk of medication errors using computerized provider order entry (CPOE) versus paper-order entry in 
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variance, also showed an increase in errors and a de- 
crease in pADEs, but statistical testing was not per- 
formed [62]. 

For two studies excluded from the pooled analysis due 
to lack of data on variance, we calculated RRs of 0.61 
[67], and 1.73, respectively [62]. In the third such study, 
the authors reported a 50% decline in medication errors 
(see Additional file 1) [57]. 

Intervention design and implementation, contextual, and 
methodological factors 

Six of the a priori subgroup analyses met the requirement 
to have at least three studies per subgroup and were, 
therefore, conducted (two were on one variable, CDSS) 
(Figure 3). Two univariate meta-regression analyses were 
able to examine whether baseline medication error rate or 
year of publication (a proxy for maturity of CPOE inter- 
vention; date of implementation was frequently missing) 
predicted effectiveness. 

Of five intervention design and implementation factors 
examined, none reached the conventional level of statis- 
tical significance, including type of developer (commer- 
cial 0.56 (95% CI 0.36 to 0.85) versus homegrown 0.37 
(0.29 to 0.47)), type of CDSS (present 0.44 (0.32 to 0.62) 
versus absent 0.51 (0.31 to 0.87), and basic 0.40 (0.38 to 
0.87) versus moderate or advanced 0.51 (0.26 to 0.97)), 
and scope of implementation (hospital-wide 0.78 (0.36 
to 1.70) versus limited 0.38 (0.32 to 0.46)). Year of publi- 
cation was not associated with differential effectiveness. 

Two contextual factors were evaluated. Studies per- 
formed in the US showed greater effectiveness than non- 
US studies, but this difference was not statistically signifi- 
cant. As the baseline percentage of hospitalizations associ- 
ated with medication errors increased from 3.6% to 99.9% 
(data available for 12 studies), the predicted RR of medica- 
tion errors with CPOE decreased from 1.90 to 0.08 (P < 
0.001). 

Regarding methodological factors, studies that used 
pharmacist order review reported greater effectiveness 
than studies using more comprehensive event detection 
methods, although this difference was not statistically 
significant. Almost all studies used pre/post designs so 
this subgroup analysis was not conducted. 

Discussion 

The principal finding of this analysis is that CPOE is as- 
sociated with a significant reduction in pADEs (hat is, 
the patient injuries it was designed to prevent) in adult 
hospital-related acute care settings. Specifically, com- 
pared with using paper orders, using CPOE was associ- 
ated with about half as many pADEs. Medication errors, 
likewise, were also about half as common with CPOE as 
with paper-order entry, and the reduction was generally 
similar across studies with different intervention designs 



and different implementation, contextual, and methodo- 
logical characteristics. There were no statistically significant 
differences in effect between commercial and homegrown 
systems, with or without CDSS of differing sophistication 
levels, and between hospital-wide or more limited imple- 
mentations. The baseline rate of hospitalizations associated 
with medication errors was significantly associated with ef- 
fectiveness, as increasing baseline rates of errors were asso- 
ciated with increasing effectiveness. This is expected, 
because, with few errors, there can be little to change. 

Our pooled analysis is conclusive that CPOE is associ- 
ated with a reduction in pADEs. Shamliyan et al. exam- 
ined ADEs that might or might not have been related to 
medication errors, and, therefore, were not as likely to be 
affected by CPOE. These authors observed significant de- 
clines in only three of seven studies (including pediatric 
ones), and did not perform a pooled analysis [37] . 

With regards to the overall pooled result for medication 
errors, our findings are generally consistent with those of 
earlier, more limited systematic reviews and meta-analyses 
[34,37,41]. Radley and colleagues also found that medica- 
tion error rates declined by about half with CPOE imple- 
mentation (48%, 95% CI 41 to 55%), using a small set of 
early studies [34]. Van Rosse and colleagues observed 
greater effectiveness with CPOE than we did (RR of medi- 
cation errors = 0.08, 95% CI 0.01 41 to 0.76), but examined 
only three diverse studies [41]. Shamliyan and colleagues 
found that CPOE was slightly more effective than we did 
(odds ratio for medication errors = 0.34, 95% CI 0.22 41 to 
0.52), based on inpatient and outpatient studies from be- 
fore 2006 [37]. In comparison to these previous studies, 
we were able to identify a greater number of relevant arti- 
cles despite having more restrictive selection criteria (see 
Additional file 1), enabling us to explore reasons for study 
heterogeneity. 

Also like previous reviews [37], we observed substantial 
variability across studies in the effectiveness of CPOE at 
reducing medication errors. It has long been suspected 
that variability in the effectiveness of a complex sociotech- 
nical intervention such as CPOE may be related not only 
to intervention design but also to context and implemen- 
tation factors [16,77,78]. However, across the intervention 
design and implementation as well as contextual variables 
that we assessed, we did not see any statistically significant 
differences in the associations between CPOE use and re- 
ductions in medication errors. Two studies of commercial 
CPOE systems in hospital-wide implementations reported 
increases in medication errors but reductions in pADEs 
[11,70]. One potential explanation for these seemingly 
contradictory results is that the CPOE systems may 
have created new medication errors at lower risk for 
causing ADEs (such as concurrent submission of dupli- 
cate orders due to order sets) but reduced medication 
errors at higher risk of causing ADEs (such as serious 
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drug- drug interactions). Alternatively, CPOE may have 
made errors easier to detect. The potential to create 
new types of low-risk medication errors calls attention 
to the importance of tailoring the CPOE system to the 
local environment because such errors place a time bur- 
den on providers. 

This analysis has limitations. We relied on 32 previous 
systematic reviews to detect primary studies published 
before 2007. Because each review detected a slightly dif- 
ferent set of publications (see Additional file 1), per- 
forming our own search of that period would have been 
unlikely to detect additional studies. We excluded 
pediatric studies instead of examining population age as 
a subgroup because these groups differ in their risk for 
experiencing medication errors and pADEs. Future in- 
vestigators could evaluate the feasibility of conducting a 
similar meta-analysis for pediatric populations. We also ex- 
cluded studies that relied upon incident reporting or did 
not describe event detection methods, considering these to 
be minimum criteria for study quality. The number of stud- 
ies that examined pADEs was not large, but all studies de- 
tected declines. Most studies were conducted in academic 
centers, limiting generalizability to community hospitals. 
Finally, the included studies all used limited methods, in- 
cluding using pre/post designs and lacking robust data- 
collection methods. 

Conclusion 

Implementing CPOE is associated with a greater than 50% 
decline in pADE rates in hospital-related settings, although 
results vary. Medication errors decline to a similar degree. 
Changes in medication errors appear to be consistent 
across commercial and homegrown systems, with or with- 
out clinical decision support, and in individual units or 
hospital-wide implementations. Many context and imple- 
mentation variables have, unfortunately, not been reported 
sufficiently to assess their association with effectiveness. 
Overall, these findings suggest that the CPOE requirements 
for meaningful use under the HITECH Act may benefit 
public health. Knowledge about how to make CPOE more 
effective would be greatly facilitated by greater reporting of 
context and implementation details. 
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